Papers with alignment metrics
Language Directions in Multilingual LLMs: A Layer-wise Diagnostic Study of Token Alignment and Pretraining Imprint (2026.acl-srw)
Copied to clipboard
| Challenge: | Using a unified probing framework, we analyze six multilingual LLMs across five languages. |
| Approach: | They analyze multilingual representations across five languages and analyze their behavior . they find that accuracy rises by +73.5 to +80.7 points from L0 to L1 on average . |
| Outcome: | The proposed framework enables a consistent and substantial early jump in accuracy across models . the token–language alignment measures where vocabulary sharing peaks . |
Entity Contrastive Learning in a Large-Scale Virtual Assistant System (2023.acl-industry)
Copied to clipboard
| Challenge: | Named Entity Recognition (NER) tasks are a well-studied and fundamental task within Natural Language Understanding (NLU). |
| Approach: | They propose to incorporate entity contrastive learning into a virtual assistant system to improve NER models by clustering similar inputs closer together in a learned representation space. |
| Outcome: | The proposed model improves against a production baseline system that does not use contrastive learning. |
A Massively Multilingual Analysis of Cross-linguality in Shared Embedding Space (2021.emnlp-main)
Copied to clipboard
| Challenge: | Cross-lingual language models house representations for many different languages in the same space. |
| Approach: | They investigate linguistic and non-linguistic factors affecting sentence-level alignment in cross-lingual pretrained language models for 101 languages and 5,050 language pairs. |
| Outcome: | The results show that word order agreement and agreement in morphological complexity are strongest predictors of cross-linguality. |
FlipGuard: Defending Preference Alignment against Update Regression with Constrained Optimization (2024.emnlp-main)
Copied to clipboard
| Challenge: | Recent advances in preference alignment have significantly improved Large Language Models' ability to generate texts that align with human preferences and values. |
| Approach: | They propose a constrained optimization approach to detect and mitigate update regression with focal attention. |
| Outcome: | The proposed approach detects and mitigates update regression with focal attention while maintaining excellent overall performance. |
Self-Correcting Text-to-Video Generation with Misalignment Detection and Localized Refinement (2026.findings-acl)
Copied to clipboard
| Challenge: | Recent text-to-video models struggle to faith-fully follow text prompts, authors say . authors propose a new refinement framework that detects fine-grained misalignments . |
| Approach: | They propose a video refinement framework that detects fine-grained misalignments . they propose preserving regions that should be preserved rather than regenerated . |
| Outcome: | The proposed framework detects fine-grained misalignments and performs targeted corrections . it preserves correctly generated entities, segments regions across frames, and regenerates problematic regions . |
Behavioral Consistency Validation for LLM Agents: An Analysis of Trading-Style Switching through Stock-Market Simulation (2026.findings-acl)
Copied to clipboard
Zeping Li, Guancheng Wan, Keyang Chen, Yu Chen, Yiwen Zhao, Philip Torr, Guangnan Ye, Zhenfei Yin, Hongfeng Chai
| Challenge: | Recent studies have applied Large Language Models (LLMs) as agents in financial stock market simulations to test if micro-level behaviors aggregate into macro-level phenomena. |
| Approach: | They propose four alignment metrics and use Mann–Whitney U tests to compare agents’ style-switching behavior with financial theory. |
| Outcome: | The proposed model is only partially consistent with financial theory. |